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COLLECTED EVALUATIONS ON THE WCPSS ALTERNATIVE SCHOOL 
PROGRAM FOR STUDENTS WITH LONG-TERM SUSPENSIONS 

Author: William L. Carruthers (850-8894) 



| EXECUTIVE SUMMARY"] 

The Wake County Public School System (WCPSS) has provided an alternative school program 
for students with long-term (LT) suspensions since the 1995-96 year. In 1995-96, the program 
was operated by the WCPSS and known as the Alternative Learning Center (ALC). Since the 
1996-97 year, die WCPSS has contracted with the Richard M. Milbum High School (RMHS), a 
private firm, to operate the program. Over these years, the WCPSS has conducted a number of 
evaluations of the program. This Executive Summary highlights major conclusions of these 
collected evaluations, including the results of two evaluations described in this report. 

Major Conclusions 

• The opportunity for students with LT suspensions to attend an alternative school provides an 
important option for these students to continue their education. After the first year, voluntary 
enrollment in the program has been at capacity, indicating the program is fulfilling a need. 

• Academic success must be the main emphasis at this form of alternative school, as basic 
academic skills are well below grade level for many students. While affective and/or 
behavioral interventions can be woven into the core curricula and teachers can develop 
helping relations with students, core curricular content must be emphasized lest students fail 
again when they return to regular school the next year. 

• Communications between the alternative school and the regular schools are critically 
important to share information on students such as academic strengths/weaknesses, 
attendance, grades, suspensions, and plans for returning to the regular schools the next year. 
Breakdowns in communication happen due to a host of factors, and it will always be 
necessary for the alternative school and regular schools to find ways to improve 
communications. 

• Students need assistance to make a successful return to the regular school. Despite the best 
efforts of the alternative school, many students experience recurring problems when they 
return to regular schools. The alternative school and regular schools both have 
responsibilities for planning and carrying out transitional support activities. 

• The WCPSS should formalize expectations with RMHS for self-evaluation in future years. 
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Overview 

This report is one in a series of four reports produced by the WCPSS Evaluation and Research 
(E&R) Department to address this particular form of WCPSS alternative school. As depicted in 
the figure below, the reports evaluate student outcomes in three different years and conduct 
follow-up studies with two different cohorts since the inception of the program in 1995-96. 



Figure 1. Chronology and Content of Alternative School Reports 



1995-96 
j ALC 


1996-97 

RMHS 


1997-98 

RMHS 


1998-99 

RMHS 


An evaluation of ALC 
outcomes in the 1995-96 year. 
E&R Report No. 97.02 


An evaluation of RMHS 
outcomes in the 1 996-97 year. 
E&R Report No. 98.02 


An evaluation of RMHS 
outcomes in the 1997-98 year. 
(This report) 




A follow-up study, evaluating outcomes for 1995-96 ALC 
participants in the 1996-97 year. 

E&R Report No. 98.09 








A follow-up study, evaluating outcomes for 1996-97 RMHS 
participants in the 1997-98 year. 

(This report) 





The remainder of this report details the results of the last two evaluations depicted in the figure 
above. In Part I of this report, results of the follow-up evaluation conducted in the 1997-98 year 
are reported for students who attended the RMHS program in the 1996-97 year. In Part II of this 
report, results are reported for the evaluation of student outcome goals that were set for the 
RMHS program in the 1997-98 year. 

Part I: RMHS 1996-97 Follow-Up Evaluation 



Part I Introduction 

The 1996-97 school year was RMHS's first year of operation in WCPSS. RMHS served 77 
students with LT suspensions in the 1996-97 year and another 28 students who were school 
dropouts or deemed at risk of school failure. This analysis is a follow-up evaluation of 
educational outcomes for students with LT suspensions in the 1997-98 year. Three outcomes 
were investigated: 

1) How many of the 77 students returned to WCPSS in the 1997-98 year; 

2) What grades did returning students earn in the 1997-98 year; and 

3) How many suspensions did returning students have in the 1997-98 year? 

Also, comparisons are made to the follow-up evaluation that was conducted with LT-suspended 
students who participated in the Alternative Learning Centers in the 1995-96 year (E&R Report 
No. 98.09). Data from the earlier evaluation were reanalyzed for this report to permit a side-by- 
side comparison of findings. 

Part I Results 

Results in the two follow-up investigations are consistent, with similar findings in both 
evaluations as described in the following text. 
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• Return Rate: Enrollment 

One finding of both reports is 
that participation in the 
alternative school program may 
have positively influenced 
students' decision to return to 
school the next year. In both 
years, alternative school 
participants returned to WCPSS 
at a substantially higher rate than 
did nonparticipants (Figure 2). 
One important caveat, however, 
is that RMHS participants may 
be different from nonparticipants 
in some unknown tendency to 
continue in an educational 
program. 



Figure 2. Cross Tabulations for Referral and 
Enrollment at Alternative School by 
Return to WCPSS the Next Year 



Returns to 
WCPSS the 
Next Year 


LT Students Referred to Attend Alternative School 


1995-96 ALC Year 
(n = 265*) 


1996-97 RMHS Year 
(n= 161**) 


Chose to enroll at ALC 


Chose to enroll at RMHS 


Yes 


No 


Yes 


No 


Yes 


82 

78.8% 


93 

57.8% 


55 

76.4% 


39 

43.8% 


No 


22 

21.2% 


68 

42.2% 


17 

23.6% 


50 

56.2% 


Significance 


Chi-square = 12.5215 
df = 1 

p < .001 


Chi-square = 17.3758 
df = 1 

p<.0001 



* Not counting 5 students who graduated in the 1995-96 year. 
** Not counting 6 students who graduated in the 1996-97 year. 



• Return Rate: Withdrawal 

One perspective on whether 
students who enrolled at an 
alternative school differed from 
those who did not enroll in terms 
of some tendency to continue in 
an educational program is to 
investigate how many of those 
who returned the next year from 
either group stayed to complete 
the year. Figure 3 shows that 
there was a significant difference 
in the 1 995-96 group, with those 
who enrolled in the ALC 
alternative program staying to 
complete the next school year at 
a higher rate than those who did 
not enroll; however, this 
difference was not apparent for 
the RMFIS students in the 1996- 
97 group. 



This finding is ambiguous but provides some 
evidence that students who enrolled in either alternative 
program may have a greater proclivity to continue in 
their educational program. 

Figure 3. Cross Tabulations for Returning Students 
Staying to Complete the School Year Following 
a LT Suspension the Year Before 



Stays to 
Complete 
the Next 
Year 


LT Students Returning to WCPSS the Next Year 


1995-96 ALC Year 
(n = 175) 


1996-97 RMHS Year 
(n = 94) 


Chose to enroll at ALC 


Chose to enroll at RMHS 


Yes 


No 


Yes 


No 


Yes 


49 

59.8% 


40 

43.0% 


26 

47.3% 


20 

51.3% 


No 


33 

40.2% 


53 

57.0% 


29 

52.7% 


19 

48.7% 


Significance 


Chi-square = 4.8892 
df = 1 
p < .05 


Chi-square = 0.1468 
Df = I 
p - ns 
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• Academic Performance 

The final grades of students 
returning to WCPSS in the year 
following their LT suspension 
were investigated. Only students 
who stayed to complete the 
school year were analyzed. As 
depicted in Figure 4, the results 
show that the two groups— 
participants and nonparticipants— 
did not differ in their subsequent 
academic success. In the year of 
the ALC program and the year of 
the RMHS program, students 
who enrolled in these programs 
were no more successful in their 
academics the next year than 
were students who did not enroll. 



One caveat to these results is that the 2x2 chi-square 
analysis shown in Figure 4 may not reveal subtle 
distinctions in academic accomplishment that might 
differ between the two groups of students. 

Figure 4. Cross Tabulations for the Academic 
Performance of Students Who Return to School the Next 
Year and Complete the Year 



Final 
Grades 
the Next 
Year 


Final Grades for Returning Students Who Complete the 
Following School Year 


1995-96 ALC Year 
(n = 86*) 


1996-97 RMHS Year 
(n = 43**) 


Chose to enroll at ALC 


Chose to enroll at RMHS 


Yes 


No 


Yes 


No 


0-49% 
Failure (Fs) 


30 

62.5% 


28 

73.7% 


15 

60.0% 


11 

61.1% 


50-100% 
Failure (Fs) 


18 

37.5% 


10 

26.3% 


10 

40.0% 


7 

38.9% 


Significance 


Chi-square = l .2082 
df= 1 
p = ns 


Chi-square = 0.0054 
Df = 1 
p = ns 



* Not counting 3 students with missing data 
** Not counting 3 students with missing data 



• Behavioral Conduct 

The number of short-term (ST) 
and LT suspensions of students 
returning to WCPSS in the year 
following their LT suspension 
was investigated. Again, only 
students who stayed to complete 
the year were analyzed. 
Regardless of whether students 
participated in the alternative 
program or not, in both years the 
percentage of students receiving 
either one or more ST 
suspensions and/or a LT 
suspension ranged between 59% 
to 75%. It is evident that many 
returning students continue to 
have behavioral difficulties. 



Some of the returning students who received LT 
suspensions in the year of their return, but stayed to 
"complete the year", were evidently re-enrolled in 
another alternative program; however, data maintained 
on the student database are not clear for a number of 
cases. 

Figure 5. Cross Tabulations for the Behavioral Conduct 
of Students Who Return to School the Next Year and 
Complete the Year 



ST + LT 
Suspensions 
the Next 
Year 


ST + LT Suspensions for Returning Students Who 
Complete the Following School Year 


1995-96 ALC Year 
(n = 89) 


1996-97 RMHS Year 
(n = 46) 


Chose to enroll at ALC 


Chose to enroll at RMHS 


Yes 


No 


Yes 


No 


None 


20 

40.8% 


15 

37.5% 


10 

38.5% 


5 

25.0% 


1 or more 


29* 

59.2% 


25** 

62.5% 


16*** 

61.5% 


15**** 

75.0% 


Significance 


Chi-square = 0. 1 0 1 5 
df- 1 
p = ns 


Chi-square = 0.9322 
df = 1 
p = ns 



* 5 students had LT suspensions 
** 2 students had LT suspensions 
*** 1 student had a LT suspension 

**** 2 students had LT suspensions 
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Part I Conclusions 

This and the previous follow-up evaluation essentially found the same results, regardless of 
whether the program was operated by the WCPSS or privately operated by RMHS. Either 
program certainly provides a continuing educational experience for those who participate in the 
year of their LT suspension, and may make a positive contribution to students’ decision to 
continue in school the next year. However, it has not been shown that a program for LT- 
suspended students makes a significant difference in the performance of students the next year 
when they return to regular schools. Students who participate in the program have about the 
same level of academic and behavioral difficulties die next year as do those who do not 
participate in the program. 



Part II: RMHS 1997-98 Evaluation of Student Outcomes 



Part II Introduction 

Following the first year of operation in 1996-97, WCPSS renewed the contract with RMHS to 
operate in the 1 997-98 year. The new contract called for a minimum of 130 students, and limited 
participation at RMHS to LT suspended students. 



Over the summer of 1997, RMHS hired a new headmaster (Elvia Walker, a former WCPSS 
principal) and a number of new staff members. The school remained at the same facility and 
opened its doors on August 1 1, 1997. The 
first student enrolled on August 18, 1997. 

A total of 140 students enrolled by the end 
of the year. Notable changes in student 
demographic characteristics at RMHS 
between the 1996-97 and 1997-98 years are 
compared in the table to the right: 

• a greater number of students enrolled in 
the 1997-98 year; 

• a higher percentage of female students 
enrolled in the 1997-98 year; 

• a higher percentage of Black students 
enrolled in the 1997-98 year; 

• a higher percentage of middle school 
students enrolled in the 1997-98 year; 
and 

• Gamer HS made more referrals and 
Millbrook HS made fewer referrals in 
the 1 997-98 year. 



1996-97 


RMHS Student 
Characteristics 


1997-98 


105 


Enrollment 


140 




Gender 




17(16.2%) 


F 


46 (32.9%) 


88 (83.8%) 


M 


94 (67.1%) 




Race 




60 (57.1%) 


Black 


95 (67.9%) 


42 (40.0%) 


White 


40 (28.6%) 


3 (2.9%) 


Other 


5 (3.6%) 




Grade 




4 (3.8%) 


6 


9 (6.4%) 


2(1.9%) 


7 


17 (12.1%) 


19(18.1%) 


8 


20 (14.3%) 


47 (44.8%) 


9 


54(38.6%) 


17(16.2%) 


10 


21 (15.0%) 


10(9.5%) 


11 


10(7.1%) 


6 (5.7%) 


12 


9 (6.4%) 




Top Four Referring 






Schools 




Millbrook HS (23) 


1st 


Garner HS (25) 


East Wake HS (15) 


2nd 


Broughton HS (9) 


Garner HS (11) 


3rd 


East Wake HS (9) 


Sanderson HS (6) 


4th 


Enloe HS (9) 
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Part II Results 

Following an evaluation of the program in the first year (E&R Report No. 98.02), objectives for 
the second year were revised and five student outcome goals were established. Results obtained 
on these five outcome goals are reported below. 

Outcome 1 : RMHS students will achieve 80% attendance. RMHS successfully achieved this 
outcome with 85.5% average daily attendance. 

Per procedures established for RMHS operations, RMHS reported student attendance 
data throughout the year to each student’s regular school where the data were to be 
entered into the Student Information Management System (SIMS). Both RMHS staff 
members and SIMS technicians at the regular schools described a variety of problems 
with this process, most of which can be grouped in one of three categories: standardized 
procedures, communication procedures, and data handling procedures. A qualitative 
analysis of their comments is reported in Figure 6 below. 



Figure 6. Communication Problems between RMHS and SIMS Technicians 



1 Issue 


Problem Concerns | 


Standardized 

Procedures 


• Principals and SIMS technicians need to be fully appraised of their responsibility 
to record RMHS alternative school data. 

• A single format for reporting information should be standardized for all schools. 


Communication 

Procedures 


• Attendance data should be communicated at regular intervals. 

• A consistent method of communicating attendance data to regular schools should 
be adopted (i.e., hard copy sent to SIMS technician with copy for principal). 


Data Handling 
Procedures 


• Attendance data should be organized in an easy-to-read format (i.e., spreadsheet). 

• A consistent set of codes should be used to designate specific events (i.e., initial 
enrollment, excused and unexcused absence, suspension, withdrawal). 

• Errors should be minimized (i.e., student ID numbers and regular schools were 
sometimes incorrect, attendance was sometimes noted for weekend dates) 



Because SIMS attendance data were not consistently available for the RMHS students, 
RMHS provided the E&R Department with attendance figures from their own records for 
the 1997-98 year. Average daily attendance was calculated by summing across all 
students the total days in attendance and the total days in membership, and dividing the 
former by the latter. Days not counted as being in attendance were absences (excused 
and unexcused) and suspensions. Average daily attendance at RMHS in the 1997-98 year 
(85.5%) was slightly greater than in the 1996-97 year (84.1%). 
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Outcome 2 : The rate at which RMHS students achieve Level III or IV on the EOG/EOC exams 
and 2.5 or greater on the Writing tests will improve by at least 50% relative to the rate RMHS 
demonstrated in 1996-97. RMHS successfully achieved this outcome with results on 38% 
(55/143) of the EOG, EOC, and Writing exams administered at or above competency standards. 

RMHS surpassed the 50% criterion for this outcome goal— achieving better than 100% 
improvement on the End-of-Grade (EOG), End-of-Course (EOC), and 7 th / 1 0 th grade 
writing exams. RMHS made this improvement with more students taking more exams 
relative to the 1996/97 year. In 1997/98, there were 78 students taking 143 exams while, 
in 1996/97, there were 43 students taking 89 exams. Results on these exams for LT- 
suspended students are reported in Figure 7 for these two years. The RMHS program 
showed noticeable improvement in the 1997-98 year in the following areas: EOG 
Reading, EOG Math, EOC English I, and EOC ELP. Areas where the RMHS program 
did not make significant improvement include EOC U.S. History and 10 th grade Writing. 

Figure 7. Comparing RMHS Students’ Competency on EOG, EOC, and Writing 

Exams for Two Years* 



Subject 


Middle School 
End-of-Grade 


High School 
End-of-Course 




Rdng 


Math 


7“ grd 
Wrtg 


Engl I 


ELP 


Blgy 


U.S. 

Hstry 


Algl 


10 lri grd 
Writing 




1 Cmptncy 
Criterion 


Level 

IH/IV 


Level 

III/IV 


Score 

2.5 


Level 

III/IV 


Level 

III/IV 


Level 

III/IV 


Level 

III/IV 


Level 

III/IV 


Score 

>= 

2.5 


Totals 


1996/97 

Exams 

Passed/ 

Taken 


4/16 


5/16 


0/11 


2/7 


0/11 


* 3/9 


0/5 


1/5 


1/9 


16/89 

(18%) 


1997/98 

Exams 

Passed/ 

Taken 


12/26 


15/27 


2/5 


9/21 


10/20 


4/11 


0/10 


3/11 


0/12 


55/143 

(38%) 



Notes: * Abbreviations are as follow: Reading (Rdng); Writing (Wrtg); English (Eng); Economics, Law, 
& Politics (ELP); Biology (Blgy); History (Hstry); Algebra (Alg); grade (grd); and Competency 
(Cmptncy). 
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Outcome .3 : RMHS students will obtain an overall growth in grade point average of at least 0.5 
points on average. RMHS just barely missed achieving this outcome; however, a lack of data on 
the mainframe database suggests caution in interpreting this finding. 

Data analysis for this outcome was conducted on: 

• students who completed the 1997-98 year at RMHS, and 

• who had final grade point averages (GPA) posted on the WCPSS mainframe database 
for the 1996-97 and 1997-98 years. 

Of the 140 students enrolled at RMHS in the 1997-98 year, one student was dropped 
from this analysis due to having no enrollment history in the WCPSS prior to the 1997-98 
year. Of the remaining 139 names submitted to the WCPSS Information Systems, final 
GPAs were available on the mainframe for only 13 students who met the two conditions 
above. Difference scores for the GPAs of these students (1997-98 final GPA minus 
1996-97 final GPA) ranged between -0.91 to +1.87. A negative difference score 
indicates a student’s GPA was worse in the second year compared to the first, and a 
positive difference score indicates the GPA was better in the second year. Difference 
scores were negative ('worse') for four students and positive ('better') for nine students. 
The sum of the difference scores for all 13 students was 5.96 and the average of the 13 
difference scores was 0.46, indicating RMHS failed to meet the criterion established for 
Outcome #3 . . .but not by much. 

The analysis of this outcome must be interpreted very cautiously for a variety of reasons. 
One reason for caution is that there was so much missing data. Also, there can be 
considerable variance in the assignment of grades between teachers at the same school 
and/or different schools. Another reason for caution is that the amount of time a student 
was enrolled at RMHS in a given course earning credit varied due to entering the school 
at different times of the year. A fourth reason for caution is that GPA data are not well 
documented on the mainframe. 
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Outcome 4 : At least 50% of 1 6 year-olds eligible to participate in the Transition-To-Work 
Program (TWP) will participate and satisfactorily complete the program. RMHS successfully 
achieved this outcome, with over 50% of eligible students participating in TWP; employers had 
positive comments about the students as well as the TWP. 

RMHS indicated there were 37 (24.1%) students among the 140 enrolled at RMHS who 
were eligible to participate in the TWP during the 1997-98 year. Of these 37 students, 13 
(86.7%) of 15 earned one unit of credit for completing the classroom portion of the TWP 
elective. Most of these students were second semester enrollees who could not complete 
enough hours of employment to satisfy the work experience portion of the TWP elective. 
Of the remaining 22 students enrolled in the TWP, 16 (72.7%) completed both the 
classroom and work portions for two units of credit. Among the total 37 students eligible 
to participate in the TWP, 29 (78.4%) satisfactorily met the classroom and/or classroom 
with work experience requirement. 

RMHS provided the names of the businesses, phone numbers, and employer contacts for 
17 of the 22 students engaged in the TWP work experience. In July 1998, the E&R 
Department was able to conduct telephone interviews with the employer contact (or 
someone who knew the student) for 14 of these students. 

• Eight of the employers interviewed knew about the TWP. These eight individuals 
indicated the TWP coordinator at RMHS had contacted them by telephone and/or 
visited the business. Five employers were new to their position and did not know 
about the TWP, and one employer indicated that the student had secured his job on 
his own rather than through the TWP. 

• The eight employers who had contact with RMHS generally had favorable comments 
to say: 

=> about the student (e.g., “ Wonderful worker ”, “ Very impressed with student ”, “/ 
was pleased with the student and his efforts ”, “The student has come a long way”, 
“ Student is a very hard worker”); 

=> about the TWP (“ Wonderful program”, “ Commend program coordinator” , 

“ Would like to have another student ”, “ Commend program coordinator and team 
for efforts”, “ TWP is an excellent program ...kids are given a chance to redeem 
themselves”). 

• Seven of the eight employers positively rated the TWP on a scale of l-to-5 (poor to 
excellent) with six 5s and one 4; one employer negatively rated the program with a 2. 

• The eight employers all indicated that students averaged at least 10 hours of work per 
week, with three students averaging 20-30 hours per week. 



11 



Carruthersl E^Milbum/Collected RMHS Reports 



Page 9 of 10 



E&R Report No. 00.01 



Collected Reports on WCPSS Alternative School 



Outcome 5 : RMHS students will have a current academic and behavioral plan, updated within 
the last month of their enrollment at RMHS. RMHS minimally met this objective with brief 
descriptions of student needs being written for most students enrolled for a month or more. 

In the original proposal to operate an alternative school, RMHS indicated that “Every 
student will have a Personalized Educational Plan (PEP) developed upon entrance to the 
school by the RMHS faculty. The student and parent or guardian will participate in the 
plan development. The PEP will serve as a contract regarding expected behavioral and 
academic performance” One finding of the RMHS evaluation for 1996-97 (E&R Report 
No. 98.02) was that there was no evidence that PEPs were implemented that year, and a 
recommendation in that report was that “RMHS needs to implement a method of 
monitoring and documenting students ’ instructional and behavioral strategies throughout 
the year for every student. ” 

In the 1997-98 year, the guidance counselor at RMHS met with students individually and 
in small groups throughout the year, and wrote descriptions of student needs for 1 16 of 
the 140 students who enrolled at RMHS. Of the remaining 24 students, 5 students were 
enrolled for less than one month, and it would be unrealistic to expect a PEP to be 
developed for students at the school for such a short period; however, there were 19 
students enrolled for periods of 1 to 7 months who were lacking any narrative 
descriptions. 

The descriptions for the 1 16 students were brief, averaging four to five lines of text per 
student. The descriptions typically incorporated some factual information pertaining to 
reasons why the student was at RMHS, some information on psycho-social characteristics 
of the student, and some information on appropriate counseling interventions (Reality 
Therapy was most often indicated). There was very little information on academic 
strategies other than occasional comments about students’ interest in particular subjects, 
having a special education classification, and/or plans to graduate and attend post- 
secondary education. None of the descriptions were dated, and there is no evidence that 
descriptions were updated or revised over time. 



Part II Conclusions 

The RMHS program met 4 of the 5 student outcome goals that were set for the school in the 
1997-98 year. It is especially notable that RMHS exceeded the goal for performance on the 
EOG/EOC/Writing exams by a factor of two-whereas the goal was for a 50% improvement 
relative to the 1996-97 year, RMHS achieved greater than 100% improvement. RMHS’ success 
on these goals may be explained by a variety of factors, but it is probable that two of the most 
important factors were the hiring of an experienced principal as headmaster and an improved 
stability in operations that a second year brings. 
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